1. Field of the Invention
The present invention relates to a motion vector detecting apparatus, and more particularly to a motion vector detecting apparatus used in a digital motion picture compression system.
2. Description of the Background Art
To transmit a huge amount of image data efficiently at high speed, the image data are compressed to reduce the data amount to be transferred. One of such image data compression method is an MPEG method that deals with motion pictures. In this MPEG method, a motion vector is detected on a picture block basis according to a block matching method. This motion vector as well as a difference value between pixels in a current picture block and in a predictive picture block is transmitted. Since the difference value is used as the data to be transmitted, if the predictive picture block and the current picture block are well-consistent (i.e., if the degree of motion is small), the difference value is small, which leads to a reduced amount of data to be transmitted. Conversely, if the predicted picture block and the current picture block are poorly consistent, the data amount to be transmitted will increase.
As the block size decreases, precision in detection will increase, but resultant transmission data amount will increase. Typically, in the block matching method in the MPEG method, a macroblock having a size of 16 pixels by 16 pixels is used as a unit block for motion vector detection. In the MPEG method, picture data are transmitted on a frame basis, each frame consisting of a plurality of macroblocks.
FIG. 42 schematically shows a construction of a divided display screen according to the block matching method. In FIG. 42, one picture IG is divided into a plurality of macroblocks MB. By way of example, FIG. 42 shows the picture that is divided into macroblocks of 4 rows by 5 columns. Macroblock MB generally has a size of 16 pixels by 16 pixels, and the number of macroblocks MB included in one picture IG is determined appropriately dependent on the number of pixels constituting picture IG.
FIG. 43 illustrates a motion vector detecting operation according to the block matching method. In FIG. 43, a search area SE having a prescribed size is determined for a current picture block CB of a target for the motion vector detection. This search area SE has a size that is predetermined both horizontally and vertically with its center being a position of macroblock (or, a template block) MB included in picture IG shown in FIG. 42. In this search area SE, from a position (a real rear point) corresponding to the position of current picture block CB, a positional vector (i, j) of a reference picture block RB included in search area SE is calculated and determined as a motion vector candidate. The correlation between current picture block CB and reference picture block RB is obtained. Typically, a sum of absolute difference values or squared difference values between pixels at corresponding positions in current picture block CB and in reference picture block RB is obtained as an estimation value. This estimation value is obtained for every positional vector in search area SE, and the displacement vector of the reference picture block having the smallest estimation value is determined as a motion vector for the current picture block CB.
When pixel data of current picture block CB is expressed as xe2x80x9caijxe2x80x9d and pixel data of reference picture block RB as xe2x80x9cbijxe2x80x9d, the estimation value is calculated from the following equation:
E=xcexa3|aijxe2x88x92bij|,
or E=(xcexa3aij2xe2x88x92bij2)
Thus, a huge number of calculations are necessary to obtain the estimation value. Moreover, after obtaining the estimation value for every reference block RB in search area SE, the motion vector should be determined, for which another huge number of calculations are required. To perform this motion vector detection at high speed, a variety of operation algorithms have been proposed. Operation algorithms in the motion picture compression system according to the MPEG method are described in the following articles.
P. Pirsch et al., xe2x80x9cVLSI Architecture for Video Compressionxe2x80x94A Surveyxe2x80x9d, Proc. IEEE Vol. 83, No. 2, pp.220-246, 1995.
M. Yoshimoto et al., xe2x80x9cULSI Realization of MPEG2 Realtime Video Encoder and Decoderxe2x80x94An Overviewxe2x80x9d, IEICE Trans. Electron., Vol. E78-C, No. 12, pp.1668-1681, 1995.
Tanaka et al., xe2x80x9cMPEG2 Encoding LSIs About To Change Audiovisual Equipment For Household Usexe2x80x9d, Nikkei Electronics, No. 711, Mar. 9, 1998.
LSIs for motion detecting operations designed to detect motion vectors at high speed are described in the following articles.
K. Ishihara et al., xe2x80x9cA-Half-Pel Precision MPEG2 Motion Estimation Processor with Concurrent Three-Vector Searchxe2x80x9d, ISSCC Digest of Technical Papers, pp.288-289, 1995.
A. Ohtani et al., xe2x80x9cA Motion Estimation Processor for MPEG2 Video Real Time Encoding at Wide Search Rangexe2x80x9d, Proc. IEEE Custom Integrated Circuits Conference, pp.405-408, 1995.
A. Hanami et al., xe2x80x9cA 165-GOPS Motion Estimation Processor with Adaptive Dual-Array Architecture for High Quality Video-Encoding Applicationsxe2x80x9d, Proc. IEEE Custom Integrated Circuits Conference, pp.169-172, 1998.
An all-sample full-search method enables the most accurate motion vector detection, in which method the above-described estimation values are calculated for all motion vector candidates within a search area. Specifically, an estimation value is obtained by calculating differences for all the pixel data in a picture block under search (i.e., a template block) and in a reference picture block. A positional vector of the reference picture block having the minimal estimation value among the estimation values estimated for all the estimation points in the search area is determined as a motion vector.
This all-sample full-search method, however, requires a huge amount of calculations and takes a long time to determine the motion vector. Thus, for high-speed motion vector detection through reduction of the computational amount, it is required to narrow the search area to reduce the number of estimation points. This means that, when a motion vector detecting apparatus is implemented with a single LSI, it is necessary to narrow a motion vector search range of this motion vector detecting apparatus of one LSI. Consequently, in order to search a motion vector over a wide range according to the all-sample full-search method, a plurality of LSIs (motion vector detecting apparatuses) should be operated in parallel. This leads to an increased number of LSIs to be used, resulting in an increase in power consumption as well as in apparatus scale.
To reduce the amount of computation in the all-sample full-search method, a variety of approaches have been proposed as follow: a sub-sampling approach, in which the differential operation is performed for only data of a part of pixels at each searching position (for each motion vector candidate, or at each estimation point); an algorithmic searching approach, in which the differential operation between a template block and a reference picture block is performed at only a part of coordinate positions in a search area according to a specific algorithm; and a combination of the sub-sampling approach and the algorithmic searching approach. A motion picture compression apparatus with a motion vector detecting circuit using the combined approach of sub-sampling and algorithmic searching is described, for example, in the following article.
M. Mizuno et al., xe2x80x9cA 1.5 W Single-Chip MPEG2 MP@ML Encoder with Low-Power Motion Estimation and Clockingxe2x80x9d, ISSCC Digest of Technical Papers, pp. 256-257, 1997.
In these motion vector searching approaches, however, a size of search area (motion vector search range) is fixed, and the motion vector is searched according to an algorithm that is statically determined independent of the characteristics of input current picture data. Therefore, such a situation may arise that, for some current picture data, it is possible to detect a motion vector having an estimation value substantially equal to that obtained using the all-sample full-search method, but for other current picture data, a motion vector having an estimation value considerably greater than that derived by the all-sample full-search method is detected, thereby hindering efficient data compression. In the MPEG method, a current picture is coded using one of the following coding schemes: intra-frame (or intra-field) predictive coding; unidirectional predictive coding in which a picture is predicted from a picture preceding in time; and bi-directional predictive coding in which a predictive picture is created using two pictures preceding and succeeding in time.
FIG. 44 shows an example of a sequence of pictures. In the MPEG method, a picture is formed of a frame or a field. Kinds of the pictures include: an I picture, which is a reference frame and in which intra-frame (intra-field) prediction is performed; a B picture which is predicted using two pictures preceding and succeeding in time; and a P picture which is predicted using a past picture. I picture is inserted every prescribed number of frames (or fields). P picture is inserted every three pictures. Among them, B pictures are inserted. In FIG. 44, I picture 401, B pictures 402 and 403, P picture 404, and B pictures 405 and 406 are illustrated. The numbers given below I, B and P indicating the characteristics of respective pictures indicate their orders in time.
For prediction of B picture 402, predictions 411 and 412 are performed. Prediction 411 is performed using I picture 401 preceding in time, and prediction 412 is performed using P picture 404 succeeding in time. B pictures are not used for prediction. Prediction 413 for P picture 404 is performed using I picture 401 preceding in time.
Between B picture 402 and P picture 404 there exists B picture 403. Therefore, the time difference (hereinafter, referred to as an xe2x80x9cinter-frame distancexe2x80x9d) between B picture 402 and I picture 401 is not the same as that between P picture 404 and B picture 402. As the inter-frame distance is larger, the degree of motion increases. Thus, it is necessary to set a prediction range as large as possible. This means that a wider search range is required for detecting the motion vector. Conversely, if the inter-frame distance is small, the degree of motion is small, and thus, an optimal motion vector can be detected using a small prediction range. Accordingly, in the case of the P picture having a long inter-frame distance for unidirectional prediction 413, it is necessary to set a search area as large as possible. On the contrary, in the case of forward (or backward) prediction in bi-directional prediction 411 or 412, it is desirable that the search area is determined taking the inter-frame distance into consideration to be allocated to each respective reference picture.
Conventionally, however, motion vector detection has been performed in a fixed manner, without taking into consideration the inter-frame distance at the time of prediction. This causes a problem that an unnecessarily large or too narrow search area is set for a current picture of a search target, which makes the motion vector detection inefficient and inaccurate.
An object of the present invention is to provide a motion vector detecting apparatus that enables efficient motion vector detection in both unidirectional and bi-directional predictions, without increasing an amount of hardware.
Another object of the present invention is to provide a motion vector detecting apparatus that improves the ratio between efficiency in motion vector detection and the amount of hardware, for both unidirectional and bi-directional predictions.
A motion vector detecting apparatus according to the present invention includes a plurality of motion vector detecting units that receive template block pixel data and search area pixel data and perform a prescribed operational process according to a block matching method to detect a motion vector for the template block according to the results of process. The plurality of motion vector detecting units include motion vector detecting units different in either or both of template block size and search area size. The template block size indicates the number of pixels to be used for motion vector estimation among the pixels in a macroblock, where the macroblock is a unit for the process according to the block matching method. The search area size represents a motion vector search range.
The motion vector detecting apparatus according to the present invention further includes: a control circuit that sets search areas for the plurality of motion vector detecting units so that the search area for the plurality of motion vector detecting units as a whole in a reference picture is varied according to characteristics of a current picture including the template block pixels; and a motion vector determining circuit that receives motion vector data from the plurality of motion vector detecting units to determine a final motion vector for the template block.
Since the search area for the plurality of motion vector detecting units as a whole is allowed to vary according to the characteristics (or the type of predictive coding) of the current picture of interest, it is possible to optimize a motion vector search range depending on the inter-frame distance. This enables efficient motion vector detection.
Furthermore, since the plurality of detecting units having different template block sizes and/or different search area sizes are used, it is possible to readily change the overall search area size as well as precision in searching. Thus, the motion vector can be detected efficiently, without increasing the amount of hardware.
The foregoing and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.